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1.  Introduction 


Teleoperated  systems  development  has  been  increasing  for  U.S.  Army  applications. 

Teleoperated  land  systems  for  route-clearing,  mine-clearing,  and  convoy  operations  have  had 
significant  increases  in  attention  by  the  U.S.  Army.  Teleoperation  is  the  remote  control  of  a 
system,  usually  via  radio  link,  that  includes  visual  and  or  other  feedback  to  “immerse”  the 
operator  into  the  remote  environment.  The  task  of  driving  is  known  to  induce  high  workload  in 
military  systems  (Wojciechowski,  2004).  Teleoperated  systems,  on  the  other  hand,  are  known  to 
have  increased  workload  as  compared  to  on-board  driving  (Draper  and  Blair,  1996;  Dixon  et  ah, 
2003;  Schipani,  2003).  Solutions  to  this  workload  have  been  sought  in  both  (1)  the  improvement 
of  control  and  display  qualities  in  the  operator  station  and  (2)  through  the  reduction  of  operator 
tasks  through  automation. 

The  U.S.  Army  is  continually  attempting  to  field  systems  that  further  remove  soldiers  from  harm 
on  the  battlefield.  For  the  job  of  mine  detection  and  route-clearing,  many  systems  have  been 
used,  but  not  many  have  attempted  to  reduce  the  workload  of  the  soldier  through  technology. 

1.1  Advanced  Robotic  Control  Modes 

The  U.S.  Army  Research  Laboratory  (ARL)  has  continually  attempted  to  find  ways  to  improve 
the  Soldier’s  performance  and  survivability  by  leveraging  technology.  In  this  case.  General 
Dynamics  Robotic  Systems  (GDRS)  adapted  an  ARL  and  U.S.  Army  Tank- Automotive 
Research,  Development,  and  Engineering  Center  (TARDEC)  control  concept  coined  “operator 
biasing”  or  “biasing”  (McDowell,  2008).  This  concept  was  originally  briefed  to  TARDEC  and 
had  gained  popularity  in  vehicle  control  systems,  as  this  method  is  being  employed  in  the 
Robotic  Mine  Detection  System  (RMDS).  The  RMDS  as  built  by  GDRS  and  Niitek 
incorporated  this  concept  into  their  system  (a  teleoperated  leader  vehicle,  Husky;  and  a  control 
station  follower  vehicle,  RG-31)  in  hopes  of  reducing  Soldier  workload  (figure  1).  Other 
methods  of  reducing  workload  were  incorporated  by  introducing  cruise  control  to  continually 
and  automatically  maintain  a  desired  vehicle  speed  to  reduce  workload  as  well. 

The  ability  to  control  teleoperated  systems  will  depend  mainly  upon  human  factors  engineering 
(HEE)  interface  design  characteristics  and  without  suitably  designed  controls  and  displays,  there 
will  be  an  additional  workload  cost  in  such  systems. 

Anecdotal  data  from  a  mine  clearing  system  using  cruise  control  suggests  that  high  workload 
was  attributed  to  the  use  of  cruise  control  in  that  system  (Haas  et  ah,  1997).  Participants  used  a 
joystick  controller  which  may  have  confused  the  operators.  Haas  also  noted  that  HEE  concerns 
for  this  type  of  slow-moving,  sensor-driven  system  will  rely  on  operator  vigilance  to  counteract 
the  high  potential  for  tedium  in  such  a  task. 
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Figure  1.  Fiusky  mine  detection  and  RG-31  control  station  vehicles. 

1.2  Precision  Navigation  Modes 

Precision  navigation  modes  are  those  that  use  previously  collected  global  positioning  system 
(GPS)  route  data  for  the  RMDS  system  to  follow.  These  are  considered  semi-autonomous 
modes  and  it  is  predicted  that  these  modes  will  create  less  workload  and  fatigue  than 
teleoperation  modes. 

Operator  biasing  is  a  precision  navigation  mode  that  allows  the  operator  to  supervise  the  progress 
of  the  teleoperated  vehicle,  occasionally  “bumping”  the  vehicles  path  to  the  right  or  left  in  small 
increments  to  adjust  for  poor  GPS  path  data.  This  method  provides  a  dead  man  algorithm  that 
requires  input  from  the  operator  every  30  s,  or  it  will  stop  the  teleoperated  system.  Operator 
biasing  will  be  referred  to  as  “Biasing”  further  in  this  report. 

1.3  Teleoperated  Systems  Performance  Literature 

Performance  of  teleoperated  systems  has  been  found  to  be  half  that  of  on-board  driving  (Scribner 
and  Gombash,  1998).  Specifically,  performance  time  to  complete  a  test  course  for  teleoperated 
driving  is  about  twice  that  of  on-board  driving  (Mitchell  et  ah,  1994;  Scribner  and  Gombash, 
1998).  The  number  of  errors  (obstacles  hit)  has  been  shown  to  be  about  double  that  of  on-board 
or  manned  driving  (Scribner  and  Gombash,  1998). 

The  field-of-view  (FOV),  depth  perception  (monocular  vs.  stereovision),  camera  resolution,  and 
camera  distortion  can  all  have  an  effect  on  operator  workload.  It  has  been  found  that  wide  FOV 
is  helpful  to  operators  in  unfamiliar  terrain  (Scribner  and  Gombash,  1998;  Voshell  et  ah,  2005; 
McGovern,  1987;  Silverman,  1982;  Kress  and  Almaula,  1988).  Smyth  et  al.  (to  be  published) 
concluded  that  a  unity  vision  display  with  an  electronically  controlled  FOV  might  be  optimal. 

To  the  contrary,  Gordon  (1966)  found  that  a  narrow  FOV  is  adequate  if  the  path  is  familiar  and 
has  no  obstacles.  Gordon  found  that  operators  could  drive  up  to  25  kph  on  a  curved  two-lane 
road  with  a  monocular  field  as  small  as  4  degrees.  It  was  concluded  that  information  derived 
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from  the  road  edges  and  eenterline  were  sufficient  for  vehicle  steering  control.  These  cues  are 
not  available  in  off-road  conditions. 

The  U.S.  Army  may  conduct  a  large  percentage  of  missions  in  off-road  and  unfamiliar  terrain 
where  wide  FOV  may  prove  to  be  a  necessity  for  situational  awareness,  navigation,  and  close 
quarters  maneuvering  (Glumm  et  ah,  1992). 

Wide  FOV  has  shown  to  aid  the  teleoperator  in  accomplishing  tasks  faster  with  fewer  errors 
(obstacles  contacted)  (Scribner  and  Gombash,  1998;  Drascic,  1991),  at  the  cost  of  increased 
motion  sickness  and  perceived  stress  (Scribner  and  Gombash,  1998;  Kress  and  Almaula,  1988). 

1,4  Motion  Sickness  in  Telerobotic  Systems 

Motion  sickness  (MS)  is  defined  as  the  physiological  response  of  the  body  when  the  visually 
perceived  movement  and  vestibular  system  sense  of  movement  receive  disparate  cues  of  motion. 
MS  is  often  the  term  used  for  various  types  of  illness  and  has  been  attributed  to  the  body’s  innate 
response  to  neurotoxin  poisoning  (Triesman,  1977).  When  the  vestibular  and  visual  systems  do 
not  have  similar  input,  the  area  postrema  of  the  brain  is  triggered  to  begin  inducement  of 
vomiting.  This  mismatch  of  system  cues  can  cause  simulator  sickness,  created  by  visual 
movement  cues  with  a  lack  of  movement  cues,  motion  or  seasickness,  caused  by  the  perceived 
vestibular  motion  without  visual  input. 

MS  in  teleoperated  systems  has  long  been  a  problem  due  to  both  the  lack  of  visual  fidelity  that  is 
associated  with  the  visual  systems,  and  the  disparate  motion  cues  from  either  a  stationary  or 
moving  command  platform. 

Tasks  on  moving  platforms  are  difficult,  and  can  elicit  motion  sickness  effects  (Cowings  et  ah, 
1999;  Hill  and  Tauson,  2005).  The  Future  Combat  System  Lead  System  Integrator  performed  a 
demonstration  in  which  the  operator  teleoperated  robotic  vehicles  from  a  moving  command 
vehicle  (Kamsickas,  2003).  The  results  showed  that  motion  made  all  tasks  harder,  compared  to 
an  exercise  in  a  simulated  environment,  and  some  tasks  (e.g.,  editing  plans  and  maps,  and  target 
acquisition)  became  almost  impossible  to  perform  due  to  the  difficulty  experienced  by  the 
operators  in  stabilizing  their  hand  movements.  The  operators  also  tended  to  over  steer  their 
robotic  vehicles  when  their  own  vehicle  was  turning  one  way  but  the  robot  needed  to  turn  the 
other  way.  Motion  also  makes  cognitive  tasks  more  challenging  (Schipani,  2003).  Schipani 
evaluated  Soldiers’  cognitive  performance  while  in  a  moving  vehicle,  and  found  significant 
accuracy  and  speed  decrements  in  performance.  Degradations  were  found  in  areas  such  as  time 
sharing,  selective  attention,  inductive  reasoning,  memorization,  and  spatial  orientation. 

The  measurement  of  motion  sickness  and  simulator  sickness  can  be  accomplished  with  one  set  of 
measures,  as  the  physiological  outcome  is  the  same  but  can  vary  widely  among  individuals. 
Several  measures  have  been  used  in  recent  literature  concerning  motion  sickness,  but  one  in 
particular  has  been  useful  in  recent  efforts.  For  this  study,  the  MS  inventory  by  Gianaros  et  al. 
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(2001)  was  selected  for  use  because  of  its  ability  to  classify  MS  further  into  specific  areas  of 
effect;  gastrointestinal,  central  nervous  system,  peripheral  nervous  system,  and  sophite-related. 

1.5  Workload  in  Telerobotic  Systems 

Performance  and  workload  have  been  assessed  for  many  different  types  of  systems.  It  has  been 
proposed  that  the  semi-autonomous  modes  of  RMDS  control  will  yield  the  least  amount  of 
operator  workload.  It  has  also  been  proposed  that  using  cruise  control  will  lower  operator 
workload  as  well.  These  are  all  logical  assumptions  that  remain  to  be  tested. 

Schipani  et  al.  (1998)  found  that  workload  increased  as  mission  distance  increased,  from  500, 
1000,  and  2000  m.  He  also  found  that  workload  increased  as  a  function  of  required  operator 
intervention  in  a  semi-autonomous  system.  The  converse  of  this,  of  course,  is  that  workload 
would  be  lower  for  higher  levels  of  autonomy. 

Glumm  et  al.  (1996)  found  that  when  using  a  computer-aided  teleoperation  (CAT)  method  for 
extending  a  teleoperated  vehicle’s  path,  that  workload  was  actually  higher  than  that  of  direct 
teleoperation.  This  may  have  been  due  to  the  distance  between  waypoints  which  was  set  at  1  m. 
This  rate  of  waypoints  for  the  speed  of  teleoperation  may  have  been  considered  high  workload. 
Speed  averages  were  7.6  and  4.7  kph  for  normal  and  CAT  modes,  respectively. 


2.  Hypotheses 


We  expected  that  manned  driving,  teleoperation,  teleoperation  with  cruise  control,  and  biasing 
would  yield  statistically  significant  performance  differences.  Specifically,  we  expected  that 
performance  would  be  significantly  better  for  manned  mode  than  for  three  remote  modes 
(teleoperation  with  or  without  cruise  control  and  biasing  modes).  We  further  expected  that 
among  remote  modes,  biasing  would  yield  performance  differences  that  were  significantly 
superior  to  both  teleoperation  modes.  These  four  modes  are  operationally  defined  in  section  3.3. 

We  expected  that  subjective  workload  ratings  would  be  substantially  higher  for  teleoperated 
remote  modes  as  compared  to  manned  driving  mode.  We  also  expected  that  workload  ratings  for 
teleoperation  modes  would  be  significantly  higher  as  compared  to  biasing  mode.  We  were 
unsure  about  differences  between  manned  and  biasing  modes. 

We  expected  that  subjective  motion  sickness  ratings  would  be  substantially  higher  for  all  remote 
modes  as  compared  to  manned  driving  mode.  We  also  expected  that  motion  sickness  ratings  for 
teleoperation  modes  would  be  significantly  higher  as  compared  to  biasing  mode.  We  also 
expected  a  decline  in  motion  sickness  ratings  over  time  (four  measurements  over  1  hr). 

This  study  proposed  to  examine  the  performance  effects  of  four  types  of  RMDS  control 
scenarios;  manned,  teleoperation,  teleoperation  with  cruise  control,  and  biasing  modes. 
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3.  Methods 


The  primary  task  in  this  study  was  a  road-edge  following  vehiele  operation  task.  Partieipants 
were  instrueted  to  operated  the  RMDS  surrogate  vehiele  as  elose  to  the  road  edge  as  possible 
while  attempting  to  maintain  a  vehiele  speed  of  12  kph. 

3.1  Participants 

Partieipants  were  seven  U.S.  Army  Soldiers  and  one  Department  of  the  Army  eivilian  employee, 
recruited  from  Fort  Belvoir,  VA.  All  participants  met  requirements  for  20/30  visual  acuity. 
Ranks  ranged  from  E-5  to  0-5.  Age  ranged  form  22  to  47  years.  Three  out  of  eight  had  robotic 
systems  experience  and  six  out  of  eight  had  remote  control  hobby  system  experience. 

3.2  Apparatus 

3.2.1  Volunteer  Agreement  Affidavit 

A  Volunteer  Agreement  Affidavit  (VAA)  (appendix  A)  was  given  to  each  test  participant  to 
review  prior  to  participating  in  the  study.  This  form  was  used  as  the  single  VAA  for  several 
studies  performed  simultaneously  which  were  all  aligned  under  one  research  protocol  number 
ARL-20078-0801 1,  entitled  “A  Comparison  of  Soldier  Performance  in  a  Moving  Command 
Vehicle  Under  Manned,  Teleoperated  and  Semi- Autonomous  Robotic  Mine  Detector  System 
Control  Modes.”  The  VAA  used  describes  this  study  and  others.  Upon  reading  the  document, 
test  participants  were  able  to  ask  all  questions  concerning  their  participation  in  the  study.  Once 
they  agreed  to  participate,  they  signed  the  document. 

3.2.2  Demographic  Questionnaire 

A  demographic  questionnaire  (appendix  B)  was  administered  to  collect  age,  gender,  MOS,  years 
in  that  MOS,  and  other  background  information. 

3.2.3  Titmus  II  Vision  Testing  Device 

Participants  were  screened  for  20/30  both-eye  visual  acuity,  far  distance  using  a  Titmus  II  visual¬ 
testing  device. 

3.2.4  ARL  Robotics  Program  Office  Fort  Indiantown  Gap  Operations  Center 

The  test  course  is  an  850-m-long  course  that  is  similar  in  nature  to  a  secondary  road.  This  is 
located  at  the  ARL  Robotics  Program  Office  facility  at  Fort  Indiantown  Gap,  PA.  This  area  has 
been  used  in  previous  tests  of  teleoperated  and  autonomous  vehicles  and  operator  performance. 
An  orange  line  was  painted  along  the  side  of  the  road  for  vehicle  tracking  purposes.  This  track 
was  driven  equally  in  left  and  right  laps  for  a  total  of  6  laps  (5100  m).  Three  laps  were  driven  to 
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the  right  and  three  to  the  left,  alternating  the  first  direetion  for  each  test  participant.  An  aerial 
photograph  shows  the  test  course  with  highlighting  in  figure  2. 


Figure  2.  Aerial  view  of  ARL  robotics  test 
course  (yellow  dashed  line)  at 
Fort  Indiantown  Gap,  PA. 

3,2,5  XUV  Teleoperated  Vehicle 

The  Experimental  Unmanned  Vehicle  (XUV)  is  a  four-wheeled  teleoperated  vehicle  outfitted 
with  visual  camera  sensors  (see  figure  3). 


Figure  3.  XUV  followed  by  moving  command  FIMMWV. 
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3,2,6  RMDS  Control  Station 


The  RMDS  control  station  is  a  variant  of  the  Route  Runner  control  station  currently  under  test 
for  teleoperated  HMMWVs.  The  control  station  was  placed  in  the  command  and  control  shelter 
of  a  HMMWV  for  operations  on  the  move.  The  video  set-up  for  this  study  was  to  use  center 
camera  view  as  the  primary  display,  and  a  vehicle  side  camera  in  the  picture-in-picture  view  for 
the  road  edge  following  task  (figure  4).  The  side  camera  used  was  chosen  for  the  corresponding 
direction  that  the  XUV  was  travelling  (right  side  for  clockwise  and  left  side  for  counter¬ 
clockwise  vehicle  operation).  In  all  experimental  trials,  the  XUV  was  followed  by  the  Command 
Vehicle  HMMWV.  The  control  station  had  a  19-in  LCD  as  its  primary  visual  display. 


Figure  4.  RMDS  surrogate  operator  control  station. 

3.2.7  Specific  Rating  of  Events  (SRE) 

The  SRE  rating  scale  (appendix  C)  (Fatkin  et  al.,  1990)  was  used  for  assessing  participant  global 
psychological  stress.  This  consists  of  a  numerical  scale  from  0  to  100  to  assess  a  person’s  stress 
at  a  specified  point  in  time. 

3.2.8  NASA  Task  Load  Index  (NASA-TLX) 

The  NASA-TLX  (appendix  D)  (Hart  and  Straveland,  1988)  was  used  to  quantify  participant 
workload  ratings  under  various  conditions.  The  NASA-TLX  has  been  validated  with 
mathematical  processing  tasks  of  various  levels  for  workload  assessment. 

3.2.9  Motion  Sickness  Assessment  Questionnaire  (MSAQ) 

The  MSAQ  (appendix  E)  (Gianaros  et  al.,  2001)  will  be  used  to  quantify  participant  workload 
ratings  under  various  conditions. 
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3.3  RMDS  Control  Modes 

The  RMDS  will  employ  a  remotely  eontrolled  Husky  vehiele  eommanded  from  an  RG3 1 
eommand  vehiele  using  an  Operator  Control  Unit  mounted  in  the  left  rear  passenger  seat. 

The  operational  RMDS  will  have  various  eamera  eonfigurations  available  ineluding  a  eenter-of- 
vehiele  mounted  eamera  (on  top  of  the  eab),  forward  looking  side  eameras  (mounted  on  the  sides 
of  the  vehicle,  low  to  see  road  edges),  and  a  rear-facing  camera  for  reverse  gear.  The  center  and 
rear  cameras  have  a  variable  Field  of  View  (FOV)  adjustment  from  22°  to  180°.  The  RMDS  will 
have  the  capability  to  be  remotely  operated  in  one  of  three  modes:  teleoperation,  teleoperation 
with  cruise  control,  and  precision-guided  mode.  The  surrogate  system  will  also  provide  on¬ 
board  or  manned  driving  as  a  baseline  comparison  to  these  remote  modes. 

These  modes  are  described  as  follows: 

•  Manned  driving:  consisted  of  driving  the  HMMWV  control  station  vehicle  around  the 
course  in  the  same  manner  as  the  remote  modes. 

•  Teleoperation:  the  remote  operation  of  a  vehicle  using  a  video  camera  and  primary  video 
display  in  conjunction  with  a  steering  wheel  and  hand  controls  for  acceleration  and 
braking.  There  is  currently  no  audio  feedback  system. 

•  Teleoperation  with  cruise  control:  the  previously  described  mode  with  the  addition  of 
continuous  speed  control  via  cruise  control,  essentially  locking  the  desired  speed  and  easily 
disengaged  with  a  cancellation  of  the  cruise  or  with  a  brake  input. 

•  Biasing:  the  XUV  semi-autonomously  follows  a  previously  recorded  GPS  path  with  the 
operator  providing:  (a)  course  corrections  (bumps)  to  correct  for  GPS  path  error  (these 
bumps  are  in  feet  per  input  via  button  on  the  steering  wheel)  and  (b)  obstacle  detection  and 
avoidance. 

3.4  RMDS  Vehicle  Operation  Scenario 

The  participants  reported  to  GDRS  and  were  provided  an  overview  of  the  study,  at  which  time 
initial  questions  could  be  asked  about  the  purpose  of  the  study  and  what  was  expected. 
Participants  were  asked  to  read  and  sign  the  informed  consent  form  if  they  agreed  to  participate. 

The  demographics  questionnaire  and  eye  exam  were  then  administered.  Participants  were  then 
asked  to  complete  the  sub-scale  comparison  phase  of  the  NASA-TLX  in  which  they  compared 
each  sub-scale  to  all  other  sub-scales  on  the  TLX  in  order  to  develop  individual  scoring  weights. 
Prior  to  any  training,  participants  received  a  safety  briefing  on  the  operation  of  teleoperated 
vehicles  and  ranges.  Participants  were  informed  that  they  could  withdraw  from  the  study  at  any 
time  for  any  reason,  especially  if  they  felt  that  they  had  become  motion  sick. 
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Following  this  training,  all  four  experimental  conditions  were  presented  to  the  participants.  There 
was  an  optional  rest  period  between  trials,  yet  none  of  the  participants  chose  to  rest  in  between 
trials.  NASA- TLX,  MSAQ,  and  SRE  were  all  administered  after  three-lap  intervals  throughout 
the  duration  of  each  experimental  trial. 

3.5  Design  and  Analysis 

The  design  of  this  experiment  was  a  single  factor  repeated  measures  design.  The  treatment 
variable  had  four  levels  based  on  type  of  control  mode. 

3.5.1  Independent  Variables 

The  variable  manipulated  in  this  study  was  the  control  mode  of  vehicle  operation  in  the  RMDS. 
The  levels  of  control  mode  were: 

•  Manned  driving  (baseline  performance) 

•  Teleoperation 

•  Teleoperation  with  cruise  control 

•  Biasing 

3.5.2  Dependant  Variables 

The  data  collected  consisted  of  both  objective  performance  of  lateral  drift  (m),  course  time  (s), 
and  subjective  ratings  of  workload,  stress,  motion  sickness,  preference  of  operational  mode,  and 
estimated  number  of  hours  of  operation  in  each  mode. 


4.  Procedure  and  Methodology 


As  part  of  the  pre-test  procedure,  participants  were  given  a  volunteer  agreement  affidavit  the 
preceding  Monday,  which  described  the  study  and  possible  risks.  They  were  then  given  a  short 
briefing  describing  the  RMDS  system  and  it’s  control  modes.  They  were  then  screened  for 
visual  acuity  using  a  Titmus  II  vision-testing  device.  Demographic  data  and  individual  NASA- 
TLX  sub-scale  weightings  were  also  collected  at  this  time.  All  participants  were  familiarized 
with  the  stress  (SRE)  and  motion  sickness  (MSAQ)  rating  questionnaires  as  well. 

All  participants  were  given  an  opportunity  to  operate  the  RMDS  surrogate  system  in  all  three 
remote  modes  for  a  minimum  of  20  min.  The  participants  reported  in  pairs  to  the  ARE 
DEMO  III  test  facility  at  Fort  Indiantown  Gap,  PA,  to  begin  study  participation. 
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Participant  Scenario 


System  familiarization  was  given  to  each  participant  prior  to  the  study  to  ensure  that  proper  and 
safe  operation  of  the  system  would  be  performed.  Participants  were  assigned  to  their  subject  ID 
numbers  and  subsequent  condition  orders  prior  to  the  experimental  data  collection  day.  The 
order  of  presentation  conditions  is  presented  in  table  1 .  The  manned  condition  is  lined-out  for 
participants  4-8  as  they  did  not  receive  this  treatment  due  to  time  constraints. 

Table  1.  Study  condition  presentation. 


Subject 

First  Scenario 

Second  Scenario 

Third  Scenario 

Fourth  Scenario 

1 

Teleop 

Teleop-cmise 

Manned 

Biasing 

2 

Teleop-cruise 

Biasing 

Teleop 

Manned 

3 

Manned 

Teleop 

Biasing 

Teleop-cmise 

4 

Biasing 

Manned 

Teleop-cmise 

Teleop 

5 

Teleop 

Teleop-cmise 

Manned 

Biasing 

6 

Teleop-cmise 

Biasing 

Teleop 

Manned 

7 

Manned 

Teleop 

Biasing 

Teleop-cmise 

8 

Biasing 

Manned 

Teleop-cmise 

Teleop 

At  the  experimental  test  site,  participants  were  given  additional  training  and  familiarization  with 
the  remote  control  modes  by  operating  up  to  two  laps  around  the  test  course.  For  each  test 
condition,  the  participants  drove  for  three  laps  as  accurately  as  they  could  while  attempting  to 
maintain  the  maximum  speed  of  12  kph  using  their  visual  display  to  track  the  vehicle  with  the 
road  edge  marking.  All  steering,  braking  and  accelerator  controls  were  performed  with  the  hands 
using  the  steering  wheel  or  controls  located  on  the  steering  wheel.  After  three  laps,  the 
participants  were  asked  to  rate  their  workload  by  assessing  the  level  of  each  NASA- TLX 
subscale,  their  stress  by  using  the  SRE  scale,  and  their  motion  sickness  with  the  MSAQ.  The 
XUV  and  control  vehicle  was  turned  around  and  the  opposite  direction  was  driven  for  another 
three  laps  with  the  same  battery  of  questionnaires  following  completion  of  the  vehicle  operation 
task.  At  the  end  of  each  condition  the  subject  was  asked  to  estimate  how  many  hours  they  could 
endure  in  that  condition,  to  the  half-hour.  The  subject  was  also  asked  to  rank  order  the 
presentation  of  all  conditions  in  order  of  personal  preference.  Test  participants  were  then  fully 
de-briefed  and  given  a  point  of  contact  for  follow-up  on  individual  performance  or  results  of  the 
study. 

All  participants  in  this  study  were  exposed  to  all  control  modes,  except  for  participants  4-8  who 
were  only  exposed  to  the  remote  control  conditions.  Time  to  complete  the  study  was  hindered 
by  poor  weather  conditions  hence  the  dropping  of  the  baseline  condition  (manned)  after  three 
participants  completed  their  experimental  runs.  Groups  of  two  soldiers  were  brought  to  the  test 
site  to  allow  for  data  collection  on  one  day.  Each  6-hr  period  included  the  following  activities: 
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An  assortment  of  workload,  stress,  and  motion  sickness  test  batteries  were  administered  to  each 
test  participant  after  both  three  and  six  laps  of  each  condition.  Under  each  condition,  test 
participants  were  asked  to  operate  the  RMDS  surrogate  system  to  follow  the  identified  road  edge 
as  accurately  as  possible  while  attempting  to  maintain  a  maximum  speed  of  12  kph.  The 
accuracy  of  the  road  edge  following  task  was  emphasized  wile  speed  maintenance  was  a 
secondary  priority,  but  nearly  equally  as  important.  Test  participants  randomly  began  the  test 
course  in  a  counter-clockwise  or  clockwise  direction  and  drove  three  laps  to  a  stop  at  the  course 
end  marker  cone.  The  XUV  and  the  control  vehicle  were  turned  around  while  mid-point 
questionnaires  were  administered.  Following  this,  the  opposite  direction  was  introduced  for 
another  three  laps.  The  test  participant  was  seated  at  a  vehicle  control  station  in  the  shelter  of  the 
control  station  HMMWV.  The  control  station  consisted  of  a  visual  display  with  vehicle  control 
touch  panel  buttons,  and  a  steering  wheel  with  brake  and  accelerator  paddles  for  hand  control  of 
these  functions.  The  steering  wheel  also  contained  buttons  for  camera  control  that  operated 
camera  zoom,  individual  camera  selection,  and  picture-in-picture  selection.  The  four  test 
conditions  together  took  --90  min  to  complete,  varying  with  weather  and  system  conditions. 

Following  the  study,  test  participants’  motion  sickness  was  assessed  to  determine  if  they  should 
remain  at  the  facility  due  to  these  effects.  None  of  the  eight  test  participants  exhibited  even  mild 
effects  of  motion  sickness  following  the  study. 

Snacks  and  beverages  were  also  made  available  throughout  the  duration  of  the  testing.  Lunch 
was  made  available  at  appropriate  times. 


5.  Results 


One-way  Analysis  of  Variances  (ANOVAs)  were  used  to  examine  the  effects  of  various  control 
modes  for  all  dependent  measures  (alpha  =  0.05  significance  level).  Tukey’s  Least  Significant 
Difference  (LSD)  was  used  as  a  post-hoc  analysis.  A  summary  of  all  ANOVAs  is  presented  in 
table  2.  The  estimated  maximum  time  of  operation  (hours)  is  presented  in  table  3.  The  rank 
order  operator  preference  of  conditions  is  presented  in  table  4. 

There  were  significant  findings  for  the  effect  of  control  mode  type  on  many  of  the  dependant 
variables.  Lateral  drift  (p  =  0.000),  course  completion  time  (p  =  0.000),  and  stress  (p  =  0.029) 
were  all  found  to  be  significant.  There  were  no  significant  differences  for  motion  sickness  rating 
data,  motion  sickness  change  scores,  or  stress  change  scores  over  time  (p  =  0.597),  (p  =  0.541), 
and  (p  =  0.336),  respectively.  The  findings  for  overall  workload  (p  =  0.058)  and  workload 
change  over  time  (p  =  0.062)  were  both  non-significant.  These  two  measures  will  be  reported 
and  discussed  as  they  demonstrate  important  trends  in  the  observed  workload  data. 
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Table  2.  ANOVA  table  of  dependant  measures. 


Condition 

SS 

df 

MS 

F 

P 

Lateral 

Drift  (m) 

Lateral  drift 

3229.24 

3 

1076.41 

61.94 

0.000 

Error 

278.24 

16.03 

17.37 

Conrse  Completion  Time  (s) 

Completion  time 

1578355.15 

3 

526118.38 

7.00 

0.003 

Error 

1201802.33 

16 

75112.64 

Overall  Workload  (NASA-TLX) 

Overall  workload 

789.05 

3 

263.01 

3.05 

0.058“ 

Error 

1379.55 

16 

86.22 

Overall  Workload  Change  (NASA-TLX) 

Overall  workload 

1357.83 

3 

452.61 

2.99 

0.062“ 

Error 

2414.67 

16 

150.91 

Motion  Sickness  (MSAQ) 

Motion  sickness 

55.00 

3 

18.36 

0.645 

0.597 

Error 

454.61 

16 

28.41 

Motion  Sickness  Change  (MSAQ 

) 

Motion  sickness  change 

47.14 

3 

15.71 

0.745 

0.541 

Error 

337.60 

16 

21.10 

Stress  (SRE) 

Overall  workload 

764.19 

3 

254.73 

0  4.12 

0.029 

Error 

803.46 

13 

61.80 

Stress  Change  (SRE) 

Motion  sickness  change 

204.36 

3 

68.12 

1.239 

0.336 

Error 

714.96 

16 

54.98 

Indicates  nearly  significant  data  that  will  be  presented  and  discussed  further. 


Table  3.  Estimated  maximum  operation  time  by  mode  type  (n  =  8). 


Operation  Mode 

Mean  Estimated  Operation  Time 
(hr) 

S.  D. 

Manned 

6.66 

2.30 

Teleoperated 

3.68 

2.29 

Teleoperated  with  cruise  control 

4.31 

2.65 

Biasing 

5.18 

3.20 

Table  4.  Rank  order  of  preferred  remote  operation  modes  (n  =  8). 


Operation  Mode 

First  Choice 

Second  Choice 

Third  Choice 

Teleop 

2 

0 

6 

Teleop  with  CC 

0 

7 

1 

Biasing 

6 

1 

1 
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The  data  for  workload,  motion  sickness,  and  stress  were  calculated  by  averaging  the  mid  and 
final  workload  scores  to  assess  an  overall  average  for  each  condition.  The  workload,  motion 
sickness  and  stress  change  scores  were  calculated  by  subtracting  the  final  scores  from  the  mid¬ 
point  scores.  The  data  for  the  significant  (and  two  nearly  significant)  results  are  depicted  in 
figures  5-9.  Boxes  in  the  data  graphics  indicate  that  significant  differences  were  found  among 
cells  for  the  post-hoc  test. 
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Lateral  Drift  (meters)  by  RMDS  Control  Mode 

Boxes  indicate  homogeneous  data  groups 


Teleoperation  Teleoperation  with  Cruise  Control 

Control  Mode 


Biasing 


Figure  5.  Lateral  drift  (m)  by  eontrol  mode  (p  =  0.000). 

The  least  amount  of  lateral  drift  was  exhibited  by  the  biasing  condition,  followed  by  manned, 
teleoperated,  and  teleoperated  with  cruise  control  (figure  5). 

Course  completion  times  showed  that  manned  driving  was  significantly  faster  than  all  the  remote 
modes  as  a  single  group  (figure  6). 

Overall  workload  ratings  yielded  significant  post-hoc  analyses  that  were  all  significantly 
different  from  each  other,  with  the  lowest  rating  for  the  biasing  condition.  Manned  operation 
showed  a  lower  rating  than  the  two  teleoperated  modes,  while  the  pure  teleoperation  mode 
yielded  the  highest  workload  ratings  (figure  7). 
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Teleoperation  Teleoperation  with  Cruise  Control  Biasing 
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Boxes  indicate  homogeneous  data  groups 


Figure  6.  Course  time  (s)  by  eontrol  mode  (p  ^  0.000). 
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Workload  Ratings  (NASA-TLX)  by  RMDS  Control  Mode 


Manned  Teleoperation  Teleoperation  with  Cruise  Control  Biasing 

Control  Mode 


Figure  7.  Overall  workload  ratings  by  eontrol  mode  (p  =  0.058.) 
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Overall  Workload  Change  Over  Time  (NASA-TLX)  by  RMDS  Control  Mode 


Figure  8.  Overall  workload  change  scores  by  control  mode  (p  =  0.062). 


Stress  Ratings  (SRE)  by  RMDS  Control  Mode 


Figure  9.  Stress  ratings  by  control  mode  (p  =  0.029). 
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Overall  workload  change  post-hoc  scores  revealed  that  different  modes  will  have  different 
change  effects  over  time.  The  notable  finding  in  this  data  was  the  very  high  positive  change  in 
workload  for  the  pure  teleoperation  mode.  The  other  interesting  finding  is  the  isolated  small 
negative  change  in  workload  for  the  biasing  condition.  This  was  the  only  negative  change  in  the 
data  (figure  8). 

Stress  ratings  were  all  significantly  different  from  each  other,  with  the  lowest  rating  for  the 
biasing  condition.  Manned  operation  showed  a  lower  rating  than  the  two  teleoperated  modes, 
while  the  pure  teleoperation  mode  yielded  the  highest  workload  ratings.  The  stress  data 
patterned  identically  with  the  workload  condition,  showing  a  high  correlation  between  stress  and 
workload  (figure  9).  The  Pearson  correlation  coefficient  was  significant  (p  =  0.01)  and 
calculated  to  be  r  =  0  .907,  df  =  23,27  for  stress  and  workload,  respectively. 


6.  Discussion 


The  first  hypothesis  of  performance  differences  for  control  mode  was  supported  by  the 
significant  differences  among  the  four  operation  modes  as  evidenced  by  the  dependant  measure 
outcomes.  The  measure  of  lateral  drift  showed  a  remarkable  level  of  significance  (p  <  0.000)  for 
this  measure.  Manned  mode  was  significantly  better  than  both  teleoperation  modes,  and  pure 
teleoperation  mode  was  superior  in  performance  to  teleoperation  with  cruise  control,  but  only  by 
about  0.01  ms,  or  1  cm.  Thus,  the  practical  difference  between  the  two  teleoperated  modes  was 
negligible.  The  surprising  outcome  was  that  biasing  mode  was  not  only  superior  to  both 
teleoperation  modes,  it  surpassed  manned  mode  by  a  large  margin.  In  retrospect,  this  may  not  be 
so  surprising,  as  the  GPS  following  capability  worked  remarkably  well  for  semi-autonomous 
vehicle  control. 

The  course  completion  time  data  showed  that  manned  driving  is  still  superior  to  all  of  the  remote 
condition  times.  In,  fact  all  of  the  remote  modes  were  within  about  40  s  of  each  other,  while  the 
manned  mode  was  better  by  a  margin  of  876  s,  or  about  14.6  min.  In  terms  of  minutes,  the 
manned  mode  required  about  21.7  min  per  trial,  while  the  remote  modes  required  about  36.4  min 
per  trial.  Course  time  data  very  closely  support  the  data  of  Scribner  and  Gombash  (1998)  and 
Mitchell  et  ah,  (1994)  who  found  that  teleoperated  time  requirements  were  about  double  that  of 
manned  operation.  It  is  interesting  to  note  that  the  speedometer  of  the  HMMWV  was  not  well- 
suited  to  display  an  accurate  speed,  as  drivers  averaged  much  faster  speeds  while  manned 
driving. 

The  second  hypothesis  of  superior  performance  for  the  manned  mode  was  also  supported  by  the 
data,  as  the  workload  for  manned  mode  was  far  lower  than  the  two  teleoperated  modes.  This 
data  supports  the  previous  work  of  Draper  and  Blair  (1996),  Dixon  et  al.  (2003),  and  Schipani 
(2003).  In  particular,  Schipani  (2003)  stated  that  workload  increased  as  a  function  of  distance. 
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This  finding  is  supported  by  the  data  for  modes  except  for  biasing  which  contradicts  Schipani’s 
findings.  In  fact,  the  biasing  mode  had  the  opposite  effect  with  a  decreasing  workload  from 
about  2500  to  5000  m  in  distance  traveled.  However,  Schipani  (2003)  is  supported  in  the  notion 
that  workload  is  increased  for  higher  levels  of  operator  involvement.  The  data  herein  do  not 
support  the  findings  of  Glumm  et  al.  (1996)  who  found  that  computer  aided  teleoperation  had 
higher  workload  than  direct  teleoperation.  It  was  found  that  the  workload  of  teleoperated  driving 
was  significantly  higher  than  teleoperation  with  cruise  control,  showing  the  obvious  benefit  of 
removing  that  task  from  the  operator’s  workload.  A  surprising  result  was  found  for  workload  in 
the  biasing  mode,  in  that  it  was  significantly  lower  than  manned  driving.  Another  interesting 
result  was  the  negative  change  score  in  overall  workload  for  the  biasing  condition.  Stress  scores 
mirrored  the  overall  workload  data  and  could  be  supported  by  similar  data  reported  by  Scribner 
and  Harper  (2001). 

The  third  hypothesis  of  operator  motion  sickness  for  remote  modes  was  not  supported,  due  to 
non-significant  differences  for  the  motion  sickness  ratings  and  motion  sickness  change  scores 
over  time. 

It  appears  that  most  of  the  significant  differences  were  yielded  for  the  measures  of  lateral  drift 
and  course  time.  Biasing  mode  clearly  held  the  widest  advantage  for  its  ability  to  allow  minimal 
lateral  oscillation  of  the  XUV.  The  differences  between  manned  and  teleoperated  modes  were 
significant,  yet  it  must  be  noted  that  the  differences  were  within  a  range  of  about  4.5  cm.  As  for 
course  completion  times,  manned  operation  was  certainly  the  clear  winner,  as  course  times  were 
several  minutes  longer  for  the  remote  modes.  Motion  sickness  had  no  discernable  effect  to  be 
measured  among  conditions,  however,  workload  ratings  and  workload  change  scores  over  time 
hold  some  very  interesting  points  to  discuss.  Workload  is  clearly  lowest  in  the  biasing  condition, 
followed  by  manned  operation,  teleoperated  with  cruise  control,  and  teleoperation.  The 
workload  for  teleoperation  was  the  highest,  clearly  as  a  result  of  the  tasks  required  with  the 
RMDS  control  station.  The  effect  of  removing  the  speed  control  task  by  adding  cruise  control 
technology  significantly  reduced  operator  workload,  while  removing  the  tasks  of  speed  and 
steering  control  altogether,  requiring  only  occasional  operator  “biasing”  inputs,  reduced  the 
workload  over  teleoperation  by  50%.  The  biasing  workload  was  the  lowest  workload  condition 
of  all. 

Operator  preference  data  clearly  showed  that  the  biasing  mode  was  preferred  from  among  all 
remote  control  modes.  The  rank  ordering  shows  that  teleoperation  with  cruise  control  and 
teleoperation  were  second  and  third  choices  for  the  participants,  respectively.  The  estimated 
number  of  operational  control  hours  was  highest  for  the  manned  condition  followed  by  biasing, 
teleoperation  with  cruise  control,  and  teleoperation.  This  shows  that  the  operators  felt  that  they 
could  operate  under  manned  conditions  the  longest  period  of  time,  followed  by  biasing,  event 
though  biasing  was  rated  lower  in  workload  than  the  manned  condition.  This  is  a  curious  result 
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that  may  be  attributable  to  operator’s  opinions  of  tasks  requiring  vigilance  (biasing)  vs.  workload 
(manned)  required  in  these  modes.  The  higher  arousal  requirement  of  manned  driving  may  be 
more  preferable  than  the  lower  arousal  requirement  of  biasing. 


7.  Conclusions 


The  data  seem  to  indicate  that  the  best  condition  to  allow  lengthy  operator  involvement  with 
simultaneous  secondary  tasks  would  be  the  biasing  condition.  This  has  caveats  associated  with 
it.  First,  the  route  to  be  cleared  would  require  that  a  “library”  of  GPS  data  exist  that  contains  the 
route  to  be  followed,  otherwise,  manned  or  teleoperation  conditions  would  be  required.  Second, 
the  workload  decline  of  the  biasing  condition  may  lead  to  operator  boredom  requiring  specific 
alerting  and  forced  input  to  reduce  the  occurrences  of  inattention  and  low  arousal,  which  the 
RMDS  system  does  currently  employ. 

The  data  also  indicate  that  if  a  Soldier  is  best  removed  from  danger  via  remote  modes,  then 
route-clearing  missions  be  performed  in  teleoperation  with  cruise  control  mode,  as  the  tradeoff 
appears  best  for  this  mode  considering  lateral  drift,  course  time,  and  workload  differences.  If 
GPS  data  is  required  for  a  new  route,  and  danger  is  low,  it  appears  that  manned  driving  would  be 
the  best  condition  based  on  lateral  drift,  workload,  and  stress  data.  Superior  speeds  cannot  be 
used  as  criteria  for  mission  selection  in  this  case  because  the  desired  speed  was  exceeded  by 
manned  operation  in  the  manned  trials  which  would  have  rendered  the  sensors  ineffective.  In 
other  words,  if  reducing  workload  while  route  clearing  in  low  danger  areas  is  a  priority,  it  would 
seem  that  manned  driving  would  be  the  best  option. 

Motion  sickness  data  in  this  study  were  non-significant  due  to  the  obvious  low-speed  application 
of  the  surrogate  system.  With  other  teleoperated  systems,  motion  sickness  appears  to  make 
significant  effects  in  speed  bands  beyond  10  mph.  As  this  system  works  at  an  optimal  speed  of 
12  kph  (7.5  mph)  this  effect  is  understandably  not  apparent  except  in  very  mild  and  rapidly 
dissipating  form  for  most  operators.  The  control  shelter  did  contain  one  window;  however,  this 
window  was  located  high  and  forward  of  the  control  station.  Its  effect  upon  motion  sickness 
scores  is  unknown. 

In  future  research  of  this  type,  it  would  be  recommended  that  various  road  surface  types  be  used 
to  examine  the  effect  for  different  control  modes.  It  is  known  that  road  surface  can  have  a  large 
effect  on  teleoperated  and  even  normal  driving  performance.  Additionally,  visual  display  factors 
such  as  field  of  view,  resolution,  and  apparent  frame  rate  could  be  examined  to  determine  their 
effects  on  operator  performance  in  these  conditions.  Other  tasks  such  as  mine  detection  sensor 
monitoring  or  communication  tasks  were  not  addressed  in  this  study  and  could  be  examined  in 
future  research. 


18 


8.  References 


Cowings,  P.  S.;  Toscano,  W.  B.;  DeRoshia,  C.;  Tauson,  R.  A.  Ejfects  of  Command  and  Control 
Vehicle  (C2V)  Operational  Environment  on  Soldier  Health  and  Performance.  Tech.  Rep. 
NASA/TM-1999-208786;  NASA  Ames;  Moffet  Field,  CA,  1999. 

Dixon,  S.  R.;  Wickens,  C.  D.;  Chang,  D.  Comparing  Quantitative  Model  Predietions  to 

Experimental  Data  in  Multiple-UAV  Flight  Control.  Proceedings  of  the  Human  Factors  and 
Ergonomics  Society  47th  Annual  Meeting,  Santa  Moniea,  CA,  2003,  pp  104-108. 

Draper,  J.  V.;  Blair,  L.  M.  Workload,  Flow,  and  Telepresence  During  Teleoperation. 
Proceedings  of  the  1996  IEEE  International  Conference  on  Robotics  and  Automation, 
Minneapolis,  MN,  April  1996. 

Draseie,  D.  Skill  Aequisition  and  Task  Performanee  in  Teleoperation  Using  Monoseopie  and 
Stereoseopic  Video  Remote  Viewing.  Proceedings  of  the  Human  Factors  Society  35th 
Annual  Meeting,  1991,2,  pp  1367-1371. 

Fatkin,  F.  T.;  King,  J.  M.;  Hudgens,  G.  A.  Evaluation  of  Stress  Experienced  by  Yellowstone 
Army  Fire  Fighters.  Teeh.  Mem.  No.  9-90;  U.S.  Army  Researeh  Faboratory:  Aberdeen 
Proving  Ground,  MD,  1990. 

Gianaros,  P.  J.;  Muth,  E.  R.;  Mordkoff,  J.  T.;  Fevine,  M.  E.;  Stern,  R.  M.  A  Questionnaire  for 
the  Assessment  of  the  Multiple  Dimensions  of  Motion  Siekness.  Aviation,  Space,  and 
Environmental  Medicine  72,  115-119. 

Glumm,  M.  M.;  Breitenbaeh,  F.  W.;  Grynovieki,  J.  O.  The  Effects  of  a  Computer-Aided 

Teleoperation  Technology  on  Operator  Workload  and  Performance  of  Concurrent  Tasks', 
ARF-TR-779;  U.S.  Army  Researeh  Faboratory:  Aberdeen  Proving  Ground,  MD,  1996. 

Glumm,  M.  M.;  Kilduff,  P.  W.;  Masley,  A.  S.  An  Experimental  Study  of  the  Effects  of  Field-of- 
View  on  Remote  Driver  Performance',  ARF-TR-25;  U.S.  Army  Researeh  Faboratory: 
Aberdeen  Proving  Ground,  MD,  1992. 

Gordon,  D.  A.  Experimental  Isolation  of  the  Driver’s  Visual  Input.  Human  Factors  1966,  8, 
I29-I37. 

Haas,  G.  A.;  Wahlde,  R.  V.;  Vong,  T.  T.;  Gombash,  J.;  Seribner,  D.  R.;  Staehowiak,  C.  S.; 

Fisher,  F.  N.  Unmanned  Ground  Vehicle  for  Mine  Detection:  Systems  Integration  Issues 
and  Recommendations',  ARF-TR-I256;  U.S.  Army  Researeh  Faboratory:  Aberdeen  Proving 
Ground,  MD,  1997. 


19 


Hart,  S.  G.;  Staveland,  L.  E.  Development  of  NASA- TLX  (Task  Load  Index):  Results  of 
Empirieal  and  Theoretical  Research.  In  Human  Mental  Workload',  Hancock,  P.  A., 

Meshkati,  N.,  Eds.;  Elsevier,  Amsterdam,  1988;  139-184. 

Hill,  S.  G.;  Tauson,  R.  A.  Soldier  Performance  Issues  in  C2  ‘On  the  Move.’  Presented  at  10th 
Int.  Command  Control  Res.  Technol.  Symp.,  http://www.dodccrp.org/events/10th_ICCRTS 
/CD/papers/074.pdf  (accessed  2005). 

Kamsickas,  G.  Future  Combat  Systems  (FCS)  Concept  and  Technology  Development  Phase  - 
Unmanned  Combat  Demonstration  -  Final  Report.  Tech.  Rep.  D786-1006102;  Boeing 
Company:  Seattle,  WA,  2003. 

Kress,  G.;  Almaula,  H.  Sensorimotor  Requirements  for  Teleoperation',  Report  R-6279;  PMC 
Corporate  Technology  Center:  Santa  Clara,  CA,  December  1988. 

McDowell,  K.  U.S.  Army  Research  Laboratory:  Aberdeen  Proving  Ground,  MD.  Notes  on 
ARE  Technology  Transition  for  Manned  and  Unmanned  Systems.  Personal  communication, 
March  2008. 

McGovern,  D.  E.  Experiences  in  Teleoperation  of  Land  Vehicles',  SAND87-1980,  UC-150; 
Sandia  National  Laboratories:  Albuquerque,  NM,  October  1987. 

Mitchell,  B.;  Yeager  E.;  Suarez,  M.;  Griffin,  J.;  Seibert,  G.  Cognitive  Strategies  in  UGV 

Navigation.  Cybernet  Systems  Corporation,  CSC-94-233-2-1;  Contract  No.  DAAA-15-93- 
C-0037;  performed  under  contract  for  U.S.  Army  Research  Laboratory:  Aberdeen  Proving 
Ground,  MD,  1994. 

Schipani,  S.  P.  An  Evaluation  of  Operator  Workload  During  Partially-Autonomous  Vehicle 
Operations.  Proceedings  ofPerMIS  2003,  retrieved  23  Pebruary  2004. 
http://www.isd.mel.nist.gov/research_areas/research_engmeermg/Performance_Metrics/Per 
MIS_2003/Proceedmgs/Schipani.pdf  (accessed  2003). 

Schipani,  S.  P.;  Bruno,  R.  S.;  Lattin,  M.  A.;  King,  B.  M.;  Patton,  D.  J.  Quantification  of 

Cognitive  Process  Degradation  While  Mobile,  Attributable  to  the  Environmental  Stressors  of 
Endurance,  Vibration,  and  Noise',  ARL-TR-1603;  U.S.  Army  Research  Laboratory: 

Aberdeen  Proving  Ground,  MD,  1998. 

Scribner,  D.  R.;  Gombash,  J.  W.  The  Effect  of  Stereoscopic  and  Wide  Field  of  View  Conditions 
on  Teleoperator  Performance',  ARL-TR-1598;  U.S.  Army  Research  Laboratory:  Aberdeen 
Proving  Ground,  MD,  1998. 

Scribner,  D.  R.;  Harper,  W.  H.  The  Effects  of  Mental  Workload:  Soldier  Shooting  and 

Secondary  Cognitive  Task  Performance,  ARL-TR-2525;  U.S.  Army  Research  Laboratory: 
Aberdeen  Proving  Ground,  MD,  2001. 


20 


Silverman,  E.  B.  Robotic  Technology  Experiments  for  Nuclear  Power  Plant  Inspection  and 
Maintenance.  Transactions  of  the  American  Nuclear  Society  1982,  43. 

Smyth,  C.  C.;  Paul,  V.;  Meldrum,  A.  M.;  McDowell,  K.  Examining  Alternative  Display 
Configurations  for  an  Indirect  Vision  Driving  Interface.  U.S.  Army  Research  Eaboratory: 
Aberdeen  Proving  Ground,  MD,  to  be  published. 

Triesman,  M.  Motion  Sickness:  An  Evolutionary  Hypothesis.  Science  1977,  197,  493-495. 

Voshell,  M.;  Woods;  Phillips,  E.  Overcoming  the  Keyhole  in  Human-Robot  Coordination: 
Simulation  and  Evaluation.  In  Proc.  Hum.  Factors  Ergonom.  Soc.  49th  Meet.',  2005;  pp. 
442-446. 

Wojciechowski,  J.  Q.  Validation  of  Improved  Research  Integration  Tool  (IMPRINT)  Driving 
Model  for  Workload  Analysis',  ARL-TR-3I45;  U.S.  Army  Research  Eaboratory:  Aberdeen 
Proving  Ground,  MD,  2004. 


21 


Intentionally  left  blank. 


22 


Appendix  A.  Volunteer  Agreement  Affidavit 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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VOLUNTEER  AGREEMENT  AFFIDAVIT: 

ARL-HRED  Local  Adaptation  of  DA  Form  5303-R.  For  use  of  this  form,  see  AR  70-25  or  AR 

40-38 


U.S.  Army  Research  Laboratory 
Human  Research  and  Engineering 
Directorate 

Aberdeen  Proving  Ground,  MD  21005 


Authority: 

Privacy  Act  of  1974,  10  U.S.C.  3013,  [Subject  to  the  authority,  direetion, 
and  eontrol  of  the  Seeretary  of  Defense  and  subjeet  to  the  provisions  of 
chapter  6  of  this  title,  the  Seeretary  of  the  Army  is  responsible  for,  and  has 
the  authority  necessary  to  conduct,  all  affairs  of  the  Department  of  the 

Army,  including  the  following  functions:  (4)  Equipping  (including 
research  and  development),  44  USC  3101  [The  head  of  each  Federal 
agency  shall  make  and  preserve  records  containing  adequate  and  proper 
doeumentation  of  the  organization,  functions,  policies,  deeisions, 
proeedures,  and  essential  transactions  of  the  ageney  and  designed  to 
furnish  the  information  necessary  to  protect  the  legal  and  finaneial  rights  of 
the  Government  and  of  persons  direetly  affected  by  the  agency's  activities] 

Principal  purpose: 

To  doeument  voluntary  participation  in  the  Researeh  program. 

Routine  Uses: 

The  SSN  and  home  address  will  be  used  for  identification  and  locating 
purposes.  Information  derived  from  the  project  will  be  used  for 
doeumentation,  adjudication  of  claims,  and  mandatory  reporting  of 
medical  conditions  as  required  by  law.  Information  may  be  furnished  to 
Federal,  State,  and  loeal  agencies. 

Disclosure: 

The  furnishing  of  your  SSN  and  home  address  is  mandatory  and  necessary 
to  provide  identification  and  to  contact  you  if  future  information  indicates 
that  your  health  may  be  adversely  affected.  Failure  to  provide  the 
information  may  preclude  your  voluntary  participation  in  this  data 
collection. 
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Part  A  •  Volunteer  agreement  affidavit  for  participants  in  approved  Department  of  Army 

research  projects 

Note:  Volunteers  are  authorized  medical  care  for  any  injury  or  disease  that  is  the  direct  result  of 
participating  in  this  project  (under  the  provisions  of  AR  40-38  and  AR  70-25). 


Part  B  •  To  be  completed  by  the  Principal  Investigator 

Note:  Instruction  for  elements  of  the  informed  consent  provided  as  detailed  explanation  in  accordance  with 


Title  of  Research  Project: 

A  Robotic  Control  Mode  Study  for  a  Robotic  Mine  Detection  System 

Human  Use  Protocol  Eog  # 
Number: 

ARL-20098-08011 

Principal  Investigator: 

David  Scribner 

Phone:  410-278-5983 

E-Mail:  dscribne@arl.army.mil 

Associate  Investigator: 

David  Dahn 

Phone:  410-935-6765 

E-Mail:  ddahn@alionscience.org 

Eocation  of  Research: 

General  Dynamics  Robotic  Systems,  Westminster,  MD 

Dates  of  Participation: 

14-31  Eebruary  2008 

Appendix  C,  AR  40-38  or  AR  70-25. 


Purpose  of  the  Research 

You  are  invited  to  partieipate  in  a  study  designed  to  evaluate  the  effeets  of  different 
control  methods  for  a  remote  control,  or  teleoperated  mine  detection  system.  More  specifically, 
the  purpose  of  this  study  is  to  compare  four  different  methods  of  control  and  their  effects  on 
route  path  quality,  course  time,  and  ratings  of  mental  effort,  motion  sickness,  and  stress.  This 
study  will  be  conducted  the  Army  Research  Laboratory  (ARL)  -  Human  Research  Engineering 
Directorate  (HRED)  at  General  Dynamics  Robotic  Systems  (GDRS)  in  Westminster,  Maryland.. 

Procedures 

Participation  in  this  study  will  require  a  one  day  of  visit  to  the  test  course  facility  at 
GDRS  in  Westminster  and  a  one-day  visit  to  the  test  facility  at  Et.  Indiantown  Gap,  PA.  On  the 
first  day,  you  will  be  asked  to  (1)  provide  written  informed  consent  to  participate  in  the  study,  (2) 
choose  whether  or  not  to  provide  the  principle  investigator  your  ASVAB  scores  (3)  be  assigned  a 
confidential  participant  ID  number,  (4)  complete  a  demographics  questionnaire,  and  (5)  be  tested 
for  visual  acuity  and  color  vision.  After  this,  you  will  be  familiarized  with  the  teleoperated 
system,  the  test  course,  and  safety  procedures  pertaining  to  the  operation  of  the  system.  You  will 
train  on  one  day  at  the  beginning  of  the  week  on  how  to  specifically  operate  all  four  modes  of 
control  for  the  teleoperated  system.  On  the  second  day,  you  will  be  exposed  to  four  experimental 
trials,  each  for  one  of  the  modes  of  control,  hollowing  this,  you  will  be  de-briefed  and  given 
information  on  how  to  contact  the  researcher  for  questions  that  you  may  have  about  your  data  or 
the  study  after  it  is  complete. 

There  is  a  risk  of  motion  sickness  in  this  study,  as  you  will  be  moving  in  a  vehicle  that 
follows  the  teleoperated  vehicle,  at  speeds  up  to  but  not  exceeding  7.5  mph.  This  is  considered  a 
slow  speed,  which  should  not  generate  incapacitating  motion  sickness.  You  are  free  to  withdraw 
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from  this  study  at  any  time  for  any  reason,  ineluding  feeling  any  effeets  of  motion  sickness.  You 
will  be  operating  inside  of  a  vehicle  that  will  be  heated  and  provide  shelter  from  weather  effects 
such  as  wind  and  precipitation.  You  will  be  asked  to  wear  a  seat  belt  at  all  times  when  operating 
the  vehicle.  You  will  also  be  asked  to  wear  a  safety  helmet  for  protection  when  inside  the 
control  vehicle. 

You  will  be  asked  to  complete  a  set  of  pre-test  questionnaires  including  a  sub-scale  comparison 
for  the  workload  and  stress  questionnaires.  You  will  be  asked  to  fill  out  workload,  motion 
sickness,  and  stress  questionnaires  after  each  5  laps  per  experimental  trial.  The  test  course  you 
will  operate  the  vehicle  on  is  approximately  600  meters  in  length.  You  will  operate  10  laps  in 
one  direction  and  then  10  in  the  other  direction.  One  run  will  be  about  an  hour  long.  You  will  be 
permitted  to  have  a  break  for  30  minutes  between  trials.  This  will  require  approximately  6  hours 
in  all.  You  may  not  be  eligible  to  participate  in  this  study  if:  (1)  your  visual  acuity  is  less  than 
20/30  when  corrected  with  glasses  or  contact  lenses,  or  if  2)  your  medical  profile  indicates  that 
your  health  status  requires  approval  by  your  physician. 

Benefits 

You  will  receive  no  benefits  from  participating  in  the  project,  other  than  the  personal 
satisfaction  of  supporting  research  efforts  to  better  understand  factors  that  affect  differences  in 
various  remote  control  modes  for  teleoperated  mine  detection  systems. 

Risks 

Risks  associated  with  this  evaluation  are  minimal  and  are  less  than  those  encountered  by  Soldiers 
during  their  normal  field  training  exercises  or  by  civilians  driving  on  public  roads.  There  is  a 
minimal  risk  of  motion  sickness  and  steps  will  be  taken  to  prepare  for  this  possibility.  These 
steps  include  having  a  motion  sickness  bag  and  pre-soaked  sterile  wipes  and  hand  sanitizer 
available  in  case  of  vomiting.  There  will  also  be  refreshments  offered  and  a  place  to  sit 
comfortably  or  lie  down  for  any  time  period  required.  If  you  develop  motion  sickness 
symptoms,  we  will  ask  you  to  remain  at  the  site  until  symptoms  disappear. 

Members  of  the  test  administration  staff  will  be  close  to  you  throughout  all  evaluation  trials  to 
assist  you  should  a  problem  arise.  If  you  ask  to  terminate  the  test,  care  will  be  taken  to  minimize 
risks  and  you  will  be  allowed  to  cease  participation.  If  the  WBGT  equals  or  exceeds  85°F  testing 
will  be  halted.  You  will  have  a  break  of  at  least  20  minutes  between  operations  conditions. 

Confidentiality 

All  data  and  information  obtained  about  you  will  be  considered  privileged  and  held  in 
confidence.  Photographic  or  video  images  of  you  taken  during  this  data  collection  will  not  be 
identified  with  any  of  your  personal  information  (name,  rank,  or  status).  Your  facial  features  and 
name  will  be  blurred  out  of  any  still  photos.  Video  footage  will  be  taken  at  such  an  angle  that 
you  will  not  be  able  to  be  positively  identified.  Complete  confidentiality  cannot  be  promised, 
particularly  if  you  are  a  military  service  member,  because  information  bearing  on  your  health 
may  be  required  to  be  reported  to  appropriate  medical  or  command  authorities.  In  addition, 
applicable  regulations  note  the  possibility  that  the  U.S.  Army  Human  Research  Protection  Office 
officials  may  inspect  the  records. 
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Disposition  of  Volunteer  Agreement  Affidavit 

The  Principal  Investigator  will  retain  the  original  signed  Volunteer  Agreement  Affidavit  and 
forward  a  photocopy  of  it  to  the  Chair  of  the  Human  Use  Committee  after  the  data  collection. 

The  Principal  Investigator  will  provide  a  copy  of  the  signed  and  initialed  Affidavit  to  you. 

Obtaining  of  ASVAB  Scores 

IF  YOU  ARE  AN  ACTIVE  DUTY  ENLISTED  MILITARY  VOLUNTEER,  we  would  like  to 
obtain  your  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  scores  for  potential  data 
analysis.  The  ASVAB  scores  would  be  used  strictly  for  research  purposes.  The  results  of  any 
such  analyses  would  be  presented  for  the  group  of  participants  as  a  whole;  and  no  names  will  be 
used.  With  your  permission,  we  will  obtain  these  scores  by  sending  a  copy  of  this  signed  consent 
form  along  with  your  Social  Security  Number  to  the  Defense  Manpower  Data  Center  (DMDC)  in 
Seaside,  CA  where  ASVAB  scores  may  be  obtained  from  their  databases  in  Arlington,  VA  or 
Seaside,  CA.  If  you  do  not  wish  your  ASVAB  scores  to  be  released  to  the  principal  investigator, 
you  will  still  be  allowed  to  participate  in  the  research. 

If  you  would  like  to  participate  in  this  research,  please  sign  one  of  the  following  statements,  and 
then  complete  the  information  requested  at  the  end  of  this  form; 

I  DO  AUTHORIZE  you  to  obtain  my  ASVAB  scores. _ 

(Your  Signature) 

I  DO  NOT  AUTHORIZE  you  to  obtain  my  ASVAB  scores. 


(Your  Signature) 

Contacts  for  Additional  Assistance 

If  you  have  questions  concerning  your  rights  on  research-related  injury,  or  if  you  have  any 
complaints  about  your  treatment  while  participating  in  this  research,  you  can  contact: 

Chair,  Human  Use  Committee  Office  of  the  Chief  Counsel 

U,S.  Army  Research  Laboratory  qr  U,S,  Army  Research  Laboratory 

Human  Research  and  Engineering  2800  Powder  Mill  Road 

Directorate  - 

Aberdeen  Proving  Ground,  MD  21005  Adelphi,  MD  20783-1197 

(410)  278-5992  or  (DSN)  298-5992  (301)  394-1070  or  (DSN)  290-1070 

I  do  hereby  volunteer  to  participate  in  the  research  project  described  in  this  document.  I  have  full 
capacity  to  consent  and  have  attained  my  18th  birthday.  The  implications  of  my  voluntary 
participation,  duration,  and  purpose  of  the  research  project,  the  methods  and  means  by  which  it  is 
to  be  conducted,  and  the  inconveniences  and  hazards  that  may  reasonably  be  expected  have  been 
explained  to  me.  I  have  been  given  an  opportunity  to  ask  questions  concerning  this  research 
project.  Any  such  questions  were  answered  to  my  full  and  complete  satisfaction.  Should  any 
further  questions  arise  concerning  my  rights  or  project  related  injury,  I  may  contact  the  ARL- 
HRED  Human  Use  Committee  Chairperson  at  Aberdeen  Proving  Ground,  Maryland,  USA 
by  telephone  at  410-278-5992  or  DSN  298-5992. 1  understand  that  any  published  data  will  not 
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reveal  my  identity.  If  I  ehoose  not  to  partieipate,  or  later  wish  to  withdraw  from  any  portion  of  it, 
I  may  do  so  without  penalty.  I  understand  that  military  personnel  are  not  subject  to  punishment 
under  the  Uniform  Code  of  Military  Justice  for  choosing  not  to  take  part  as  human  volunteers 
and  that  no  administrative  sanctions  can  be  given  me  for  choosing  not  to  participate.  I  may  at  any 
time  during  the  course  of  the  project  revoke  my  consent  and  withdraw  without  penalty  or  loss  of 
benefits.  However,  I  may  be  required  (military  volunteer)  or  requested  (civilian  volunteer)  to 
undergo  certain  examinations  if,  in  the  opinion  of  an  attending  physician,  such  examinations  are 
necessary  for  my  health  and  well  being. 


Printed  Name  Of  Volunteer  (First,  ML,  Last) 

Social  Security  Number 
(SSN) 

Permanent  Address  Of  Volunteer 

Date  Of  Birth 
(Month,  Day,  Year) 

Today’s  Date 
(Month,  Day,  Year) 

Signature  Of  Volunteer 

Signature  Of  Administrator 
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Appendix  B.  Demographic  Data  Collection  Form 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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DEMOGRAPHICS  AND  EXPERIENCE  QUESTIONNAIRE  Soldier  ID _ 

Age _ Height _ ft _ in  Weight _ lbs 

Rank _ Date  entered  military  (month) _ (year) _ 

Date  Armed  Serviees  Voeational  Aptitude  Battery  (ASVAB)  taken  (month) _ 

(Year) _ ^N/A _ 

If  applieable,  Primary  MOS _  Seeondary  MOS _  OR  Job 

T  itle _ 

1 .  Have  you  operated  a  remote  military  system  before?  _ Yes  _ ^No 

2.  Have  you  operated  radio-eontrolled  hobby  systems  before?  (Car,  plane)  _ ^Yes  _ ^No 

3.  How  well  do  you  feel  you  perform  with  a  remote  vehieles? 

_ Poor  _ Below  Average  _ Average  _ ^Above  Average  _ Exeellent 

4.  Does  your  Military  Oeeupational  Speeialty  inelude  driving  any  vehieles?  _ Yes  _ ^No 

5.  Are  you  _ left  handed  or  _ right  handed? 

6.  Do  you  use  your _ left  eye  or _ right  eye  to  aim  a  weapon? 

7.  Do  you  wear  glasses/contact  lenses  when  you  drive  ? _ Yes  _ No 

8.  a.  Do  you  play  video  games  or  eomputer  games?  _ ^Yes  _ ^No 

b.  What  type  of  speeifie  systems  do  you  use?  _ Console  _ PC  _ Both 

9.  Do  you  ever  play  simulations  or  games  that  have  driving  involved?  _ ^Yes  _ ^No 

10.  How  well  do  you  think  you  play  driving  video  games? 

_ Poor  _ Below  Average  _ ^Average  Above  Average  _ Exeellent 

1 1 .  What  is  your  eurrent  edueation  level? 

_ High  School  _ Junior  College  _ Bachelor’s  Degree  _ MA  _ PhD 

12.  How  susceptible  are  you  to  motion  sickness? _ Eow  _ Moderate  _ high 


Vision  Testing  Score:  (Acuity) 


(Color  Vision) 
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Appendix  C.  Specific  Rating  Of  Events  (SRE)  Form 


This  appendix  appears  in  its  originai  form,  without  editoriai  change. 


Soldier  ID;  _ 

Date: _ 

Condition: _ 

PRE-EXPERIMENTAL  RUN  IBASELINE) 

1 .  The  seale  below  represents  a  range  of  how  stressful  an  event  might  be.  Put  an  “X”  on  the  line 
to  rate  how  mueh  stress  you  experieneed  right  now?. 


Not  at  All 
Stressful 

1 _ 

Most  Stress 
Possible 

_ 1 

0 

10 

20 

30 

40 

50 

60 

70 

80 

90 

1 

100 

2.  At  what  number  value  does  the  “X”  toueh  the  line? 
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Soldier  ID;  _ 

Date: _ 

Condition: _ 

PRE-EXPERIMENTAL  RUN 

1 .  The  seale  below  represents  a  range  of  how  stressful  an  event  might  be.  Put  an  “X”  on  the  line 
to  rate  how  mueh  stress  are  you  experieneing  eurrently  during  this  vehiele  operation?. 

Not  at  All  Most  Stress 

Stressful  Possible 


0  10  2  0  3  0  40  50  60  7  0  8  0  90  100 

2.  At  what  number  value  does  the  “X”  toueh  the  line? 
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Intentionally  left  blank. 
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Appendix  D.  NASA  Task  Load  Index  (NASA-TLX) 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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NASA-TLX  Workload  Quesitonnaire  (Weighting  Selection) 


Soldier  ID _ 

Date  /  / 


(one  time  only,  per  test  participant) 


For  each  pair,  select  the  one  element  that  is 
more  important  for  measurement  of  workload 
for  the  task. 


1  1  Mental  Demand  / 

Physical  Demand 

□ 

1  1  Mental  demand  / 

Temporal  Demand 

1  1  Mental  Demand  / 

Performance 

□ 

1  1  Mental  Demand  / 

Effort 

□ 

1  1  Mental  Demand  / 

Frustration 

□ 

1  1  Physical  demand  / 

Temporal  Demand 

□ 

1  1  Physical  Demand  / 

Performance 

□ 

1  1  Physical  Demand  / 

Effort 

□ 

1  1  Physical  Demand  / 

Frustration 

□ 

1  1  Temporal  Demand  / 

Performance 

□ 

1  1  Temporal  Demand  / 

Effort 

□ 

1  1  Temporal  Demand  / 

Frustration 

□ 

1  1  Performance  / 

Effort 

□ 

1  1  Performance  / 

Frustration 

□ 

□  Effort  / 

Frustration 

□ 
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NASA-TLX  RATING  SCALE  DEFINITIONS 

MENTAL  DEMAND  Low/High  How  much  mental  and  perceptual  activity 

was  required  (e.g.,  thinking,  deciding,  calculating, 
remembering,  looking,  searching,  etc.)?  Was  the  task  easy 
or  demanding,  simple 
or  complex,  exacting  or  forgiving? 

PHYSICAL  DEMAND  L  o  w/High  How  much  physical  activity  was  required 

(e.g..  pushing,  pulling,  turning,  controlling, 
activating,,  etc.)?  Was  the  task  easy  or 
demanding,  slow  or  brisk,  slack  or  strenuous 
restful  or  laborious? 


TEMPORAL  DEMAND  L  o  w/Hig/;  How  much  time  pressure  did  you  feel  due  to 

the  rate  or  pace  at  which  the  tasks  or  task 
elements  occurred?  Was  the  pace  slow  and 
leisurely  or  rapid  and  frantic? 

PEREORMANCE  Poor/Good  How  successful  do  you  think  you  were  in 

accomplishing  the  goals  of  the  task  set  by 
the  experimenter  (or  yourself)?  How 
satisfied  were  you  with  your  performance  in 
accomplishing  these  goals? 

EFFORT  Low/High  How  hard  did  you  have  to  work  (mentally 

and  physically)  to  accomplish  )'our  level  of 
performance? 

FRUSTRATION  LEVEL  Low/High  How  insecure,  discouraged,  irritated,  stressed 

and  annoyed  versus  secure,  gratified,  content, 
relaxed  and  complacent  did  you  feel  during  the  task?  . 
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NASA-TLX  Workload  Questionnaire  (Weighting  Selection) 


Soldier  ID _ 

Date _ 

Condition _ 

For  each  workload  element  listed  below,  please  indicate  (with  an  exact  mark  on  the  line)  how  much 
each  element  contributed  to  your  overall  workload  experienced  in  the  task  you  just  performed.  Please 
write  the  corresponding  number  for  your  mark  in  the  space  provided  below  each  line. 


Mental  Workloald — i - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - l 

0 

Low 

What  number  is  this? 


100 

High 


Physical  deman 


i- 


J _ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ _ L 


0 

Low 


100 

High 


What  number  is  this? 


Temporal  Demahit 


J _ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ _ L 


0 

Low 


What  number  is  this? 


100 

High 


Performance 


0 

Low 

What  number  is  this? 


100 

High 


Effort 


0 

Low 

What  number  is  this? 


100 

High 


Frustration 


0 

Low 

What  number  is  this? 


100 

High 
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Appendix  E.  Motion  Sickness  Assessment  Questionnaire  (MSAQ)  Form 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Motion  Sickness  Assessment  Questionnaire 


Soldier  ID 


Date _ 

Condition: _ 

Using  the  scale  below,  pleasecircle  the  number  that  rates  how  accurately  the  following  statements  describe  your 
experience. 


1 . 1  felt  sick  to  my  stomach 

0  T  ■fplt  fcutTf  llW'P 

Not  at  all 

1 

0 

_ 4__ 

6- 

_ 8_ 

Severely 

Q 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

— 8— 

y 

Severely 

....Q 

J.  J.dl  J-dllll  llJvC 

3 . 1  felt  annoyed  /  irritated 

4. 1  felt  sweaty 

Not  at  all 

1 _ 2— 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

— 8— 

y 

Severely 

....Q 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

— 8— 

y 

Severely 

....Q 

Not  at  all 

1 _ 2— 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

— 8— 

y 

Severely 

....Q 

J.  1  iCll  mJCaay 

6. 1  felt  lightheaded 

7. 1  felt  drowsy 

8. 1  felt  clammy  /  cold  sweat 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

..8... 

y 

Severely 

....Q 

Not  at  all 

1 _ 2— 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

..8... 

y 

Severely 

....Q 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

..8... 

y 

Severely 

....Q 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

..8... 

y 

Severely 

....Q 

y.  1  icii  uiouiiciiicu 

10. 1  felt  tired  /  fatigued 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

..8... 

y 

Severely 

....Q 

Not  at  all 

1 _ 2— 

_ 0___ 

_ 4 _ 

u 

_ 

0 

— R— 

y 

Severely 

....Q 

i  i  .  1  iCll  IlaUoCdlCU. 

10  T  ■fplf"  /  \x7ctTm 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

— R— 

y 

Severely 

....Q 

LZ..  1  iCll  llUl  /  Wallll 

13.1  felt  dizzy 

1  A  1  tol'f  lilz'fa  1  wrctc  CMi 1  rr 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 

0 

— R— 

y 

Severely 

....Q 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

— R— 

y 

Severely 

....Q 

It-.  1  iCll  llJvC  i  Wdo  oUllllllll^ 

15.1  felt  as  if  I  may  vomit 

16. 1  felt  uneasy 

Not  at  all 

1 _ 2— 

D 

_ 0___ 

_ 4 _ 

u 

_ 6— 

0 

— R— 

y 

Severely 

....Q 

Not  at  all 

1 - 2— 

D 

....3... 

____4____ 

-5  - 

u 

....6— 

....7..... 

0 

-8— 

y 

Severely 

....9 

17.  How  many  times  have  you  vomited  today? 
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NO.  OF 

COPIES  ORGANIZATION 


1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
only)  DTIC  OCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BELVOIR  VA  22060-6218 

1  US  ARMY  RSRCH  DEV  & 
ENGRG  CMD 
SYSTEMS  OF  SYSTEMS 
INTEGRATION 
AMSRD  SS  T 
6000  6TH  ST  STE  100 
FORT  BELVOIR  VA  22060-5608 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  IMS 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARE  Cl  OK  TL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARE  Cl  OK  PE 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 


ABERDEEN  PROVING  GROUND 
1  DIR  USARL 

AMSRD  ARE  Cl  OK  TP  (BLDG  4600) 
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NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRDARLHRML  J  MARTIN 
MYERCTR  RM2D3II 
FT  MONMOUTH  NJ  07703-5601 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRDARLHRMZ  A  DAVISON 
320  MANSCEN  LOOP  STL  1 15 
FT  LEONARD  WOOD  MO  65473 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRDARLHRMD  T  COOK 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 

1  COMMANDANT  USAADASCH 
ATSA  CD 

AMSRDARLHRML  DR  HAWLEY 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MM  DR  V  J  RICE 
BLDG  4011  RM2I7 
1750  GREELEY  RD 
FT  SAM  HOUSTON  TX  78234-5002 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MG  R  SPINE 
BLDG  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARE  HRED  ARMC  FED  ELMT 

AMSRD  ARE  HR  MH  C  BURNS 
BLDG  1467B  RM  336 
THIRD  AVE 
FT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 
AWC  FIELD  ELEMENT 
AMSRD  ARE  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MV  HQ  USAOTC 
S  MIDDLEBROOKS 
91012  STATION  AVE  RM  348 
FT  HOOD  TX  76544-5073 


1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MY  M  BARNES 
2520  HEALY  AVE 
STL  1172  BLDG  51005 
FTHUACHUCAAZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MP  DUNGVARSKY 
POPE  HALL  BLDG  4709 
BCBL  806  HARRISON  DR 
FT  LEAVENWORTH  KS  66027-2302 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MJF  J  HANSBERGER 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PKWY  STL  B 
SUFFOLK  VA  23435 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MQ  MR  FLETCHER 
US  ARMY  SBCCOM 
NATICK  SOLDIER 
AMSRD  NSC  WSE  BLDG  3  RM  343 
NATICK  MA  01760-5020 

1  ARMY  RSCH  LABORATORY-HRED 
AMSRD  ARE  HR  MT  J  CHEN 
12423  RESEARCH  PKWY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MT  C  KORTENHAUS 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MS  C  MANASCO 
SIGNAL  TOWERS 
BLDG  29808A  RM  303 
FT  GORDON  GA  30905-5233 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARE  HR  MU  M  SINGAPORE 
6501  E  11  MILE  RD  MS  284 
BLDG  200 A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRDARLHRML  C  HERNANDEZ 
2421  NW  AUSTIN  RD  STL  220 
FT  SILL  OK  73503-9042 
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NO.  OF 

COPIES  ORGANIZATION 

1  ARMY  RSCH  LABORATORY  -  HRED 

AMSRDARLHRMW  E  REDDEN 
BLDG  4  ROOM  332 
FTBENNINGGA  31905-5400 

1  ARMY  RSCH  LABORATORY  -  HRED 

AMSRD  ARE  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FT  BRAGG  NC  28310-5000 

I  ARMY  GI 
(CD  DAPE  MR  B  KNAPP 
only)  300  ARMY  PENTAGON  ROOM  2C489 
WASHINGTON  DC  20310-0300 


ABERDEEN  PROVING  GROUND 

2  DIR  USARL 

AMSRD  ARE  Cl  OK  TP 
S  FOPPIANO 
AMSRD  ARE  HR  MR 
F  PARAGALLO 
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NO.  OF 

COPIES  ORGANIZATION 


10  GENERAL  DYNAMICS 
ROBOTICS  SYSTEMS 
DDAHN 
1231  TECH  CRT 

WESTMINSTER  MD  21157-3029 


ABERDEEN  PROVING  GROUND 

20  DIR  USARL 

AMSRD  ARE  HR  MB 
D  SCRIBNER 
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